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Abstract.  Studies  in  recent  years  have  demonstrated  that  neural  organization 
and  structure  impact  an  individual’s  ability  to  perform  a  given  task.  Specifical¬ 
ly,  more  efficient  functional  networks  have  been  shown  to  produce  better  per¬ 
formance.  We  apply  this  principle  to  evaluation  of  a  working  memory  task  by 
providing  two  novel  approaches  for  characterizing  functional  network  connec¬ 
tivity  from  electroencephalography  (EEG).  Our  first  approach  represents  func¬ 
tional  connectivity  structure  through  the  distrihution  of  eigenvalues  making  up 
channel  coherence  matrices  in  multiple  frequency  hands.  Our  second  approach 
uses  a  connectivity  matrix  at  each  frequency  hand,  assessing  variability  in  aver¬ 
age  path  lengths  and  degree  across  the  network.  We  also  use  features  based  on 
the  pattern  of  frequency  band  power  across  the  EEG  channels.  Eailures  in  digit 
and  sentence  recall  on  single  trials  are  detected  using  a  Gaussian  classifier  for 
each  feature  set  at  each  frequency  hand.  The  classifier  results  are  then  fused 
across  frequency  bands,  with  the  resulting  detection  performance  summarized 
using  the  area  under  the  receiver  operating  characteristic  curve  (AUG)  statistic. 
Eused  AUC  results  of  0.63/0.58/0.61  for  digit  recall  failure  and  0.57/0.59/0.47 
for  sentence  recall  failure  are  obtained  from  the  connectivity  structure,  graph 
variahility,  and  channel  power  features  respectively. 


1  Introduction 

Recent  studies  have  investigated  neural  efficiency  as  a  way  of  measuring  an  individu¬ 
al’s  ability  to  perform  a  specific  task.  Neural  efficiency  was  examined  by  Neubauer, 
where  he  showed  that  individuals  with  a  higher  functional  ability  also  showed  lower 
event  related  desynchronizations  [11]  while  performing  the  task.  This  principle  has 
been  employed  along  with  network  analysis  to  represent  the  changes  in  the  properties 
of  the  functional  connectivity  network  [6,  8,  9].  This  has  resulted  in  studies  demon¬ 
strating  increased  task  proficiency  is  exhibited  by  a  network  exhibiting  more  “small 
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world”  properties  (i.e.  where  the  network  has  a  higher  clustering  coefficient  and  lower 
average  shortest  path  length).  In  this  paper,  we  examine  changes  in  network  proper¬ 
ties  during  a  working  memory  task,  and  provide  a  comparison  using  alternate  ap¬ 
proaches  to  characterizing  network  connectivity,  and  a  non-network  based  approach 
that  assess  changes  in  power  across  a  set  of  frequency  bands. 


2  Materials 

This  work  leverages  a  previously  collected  database  published  in  [1].  This  database 
provides  an  IRB  approved  working  memory  protocol  where  subjects  are  asked  to 
retain  a  series  of  digits,  followed  by  a  sentence  in  memory.  After  a  pause  period  the 
subjects  were  asked  to  repeat  the  memorized  sentence,  and  then  to  repeat  the  memo¬ 
rized  digits.  A  trial  was  defined  as  the  memorization  and  repetition  of  one  sentence, 
and  one  set  of  digits.  Performance  was  identified  based  on  whether  the  subject  could 
correctly  repeat  the  entire  sentence  (sentence  performance),  and  the  entire  sequence  of 
digits  (digit  performance). 

During  this  protocol,  speech,  video  data,  and  electroencephalography  (EEG),  were 
collected.  This  paper  presents  an  analysis  of  the  predictive  performance  of  EEG  in 
identifying  sentence  performance  and  digit  performance  on  a  per  trial  basis.  EEG  was 
collected  on  14  subjects,  where  each  subjects’  data  were  preprocessed  with  a  highpass 
filter  at  0.1  Hz,  a  notch  filter  at  60  Hz,  and  had  blinks  removed  using  independent 
component  analysis. 


3  Methods 

3.1  Coherence  as  a  Measure  of  Functional  Connectivity 

In  order  to  obtain  sensitive  multivariate  measures  of  frequency-band  dependent  com¬ 
munication  between  brain  regions,  we  utilize  features  based  on  the  full  set  of  signal 
coherences  obtained  between  channels  within  previously  used  neurologically  repre¬ 
sentative  frequency  bands  (theta,  alpha,  beta,  gamma)  [4].  The  coherence  between 
channels  indicates  the  amount  of  cross-channel  power  in  a  frequency  band  relative  to 
the  amount  of  within-channel  power.  This  provides  a  measure  of  how  closely  related 
the  signals  are  within  a  given  frequency  band. 
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Fig.  1.  Coherence  matrices  for  the  same  subject  on  a  trial  with  correct  (left)  and  incorrect  (mid¬ 
dle)  digit  recall,  (right)  rank-ordered  eigenvalues  from  the  two  coherence  matrices. 

Specifically,  channel-pairwise  coherences  are  computed  to  yield  a  64x64  coher¬ 
ence  matrix  for  each  frequency  band  in  each  trial.  Figure  1  shows,  for  example,  co¬ 
herence  matrices  (in  the  Theta  band)  from  a  single  subject  in  two  nearby  trials  from 
the  middle  of  the  subject’s  2-hour  session.  The  matrix  on  the  left  corresponds  to  a  trial 
when  the  subject  reported  the  four  digits  correctly,  and  the  matrix  on  the  right  corre¬ 
sponds  to  a  trial  when  they  reported  the  digits  incorrectly.  There  is  a  striking  differ¬ 
ence  in  the  appearance  of  the  two  coherence  matrices,  with  the  incorrect  trial  showing 
a  larger  number  of  strong  pairwise  coherences. 


3.2  Connectivity  Structure 


To  obtain  features  that  are  sensitive  to  changes  in  the  overall  structure  of  coherences 
among  EEG  channels,  while  being  invariant  to  the  particularities  of  coherence  pat¬ 
terns  among  specific  channel  pairs,  we  propose  the  use  of  coherence  structure  features 
[1].  Coherence  structure  refers  to  the  distribution  of  eigenvalues  in  a  coherence  ma¬ 
trix,  which  encodes  the  overall  shape  of  the  multivariate  coherence  distribution,  invar¬ 
iant  to  axis  assignments.  Eigure  1  (right)  plots  the  coherence  structure  features,  which 
are  the  rank-ordered  eigenvalues  from  the  two  coherence  matrices,  illustrating  that  the 
digits  correct  case  (blue)  produces  greater  power  in  the  small  matrix  eigenvalues  than 
the  incorrect  case  (red),  thereby  indicating  a  more  isotropic  pattern  of  coherences 
during  correct  performance,  which  we  take  as  an  indicator  of  greater  complexity  in 
cortical  communication. 

The  coherence  structure  patterns  in  Eigure  1  depict  two  trials  from  a  single  subject. 
How  do  these  patterns  generalize  to  all  the  trials  and  subjects  in  the  dataset?  To  an¬ 
swer  this  question,  we  first  normalize  (z-score)  the  eigenvalues  at  each  rank  across  all 
the  trials  in  the  data  set,  and  then  plot  in  Eigure  2  the  average  of  these  normalized 
eigenvalues  for  the  digit  correct  (blue)  and  digit  incorrect  (red)  cases  for  the  four  fre¬ 
quency  bands.  Eigure  2  shows  that  a  similar  difference  is  found  across  all  four  fre¬ 
quency  bands,  with  an  interesting  leftward  shift  in  the  eigenvalue  differences  as  we 
move  to  higher  frequency  bands. 


Fig.  2.  Averages,  computed  across  all  trials,  of  normalized  eigenvalues  for  digit  correct  (blue) 
and  digit  incorrect  (red)  recall  in  four  frequency  bands. 

3.3  Graph  Measures  of  Functional  Connectivity 

Recent  work  has  considered  network  (or  graph)  features  of  brain  activity.  This  section 
de-fines  the  construction  process  for  the  functional  connectivity  graph  and  the  fea¬ 
tures  used  in  the  classifier. 

A  graph  G=(V,  E)  is  a  pair  of  sets:  a  set  of  vertices,  V,  representing  entities,  and  a 
set  of  edges,  E,  representing  connections  or  interactions  between  the  entities.  In  the 
context  of  this  paper,  the  vertices  are  EEG  channels  and  the  edges  denote  functional 
synchrony  between  the  channels.  This  is  measured  by  coherence  between  the  meas¬ 
urements  in  the  channels,  as  described  in  Section  3.1. 

The  coherence  values  are  stored  in  a  matrix,  which  is  thresholded  to  define  undi¬ 
rected  connections  (E)  between  nodes  (V).  A  set  of  thresholds  are  used,  where  each 
threshold  is  set  to  keep  a  constant  density  (as  defined  below).  This  approach  builds  off 
[7],  where  density  is  controlled  to  keep  a  similar  number  of  connections  across  sub¬ 
jects.  However,  instead  of  using  one  threshold  as  done  in  [7],  we  aggregate  features 
across  thresholds  and  then  use  dimensionality  reduction  to  remove  redundancies. 

After  thresholding  each  coherence  matrix  to  form  a  graph,  the  following  features 
are  extracted  for  classification. 


Density.  The  proportion  of  possible  edges  that  exist  in  the  graph,  |£'|/('2')- 
is  used  for  normalization  of  the  data:  each  graph  has  equal  density. 

Average  path  length.  Eor  a  given  vertex  v,  the  average  distance  from  v  to  any  other 
vertex  in  the  graph, 

=  (1) 
uev 


Average  path  length  is  small  in  small-world  graphs. 


Degree.  The  number  of  edges  adjacent  to  a  given  vertex,  denoted 

K^\[u\{u,v]eE]\  (2) 

Small-world  networks  tend  to  have  some  “hub”  vertices  with  high  degree,  to 
facilitate  short  paths  through  the  network. 

As  in  [2],  we  consider  statistics  of  the  vertex -based  features  (average  path  length, 
and  degree)  across  all  vertices  in  the  graph.  Specifically,  we  consider  the  maximum 
value,  the  mean,  and  the  standard  deviation.  This  provides  information  about  the  dis¬ 
tribution  of  features  across  vertices  without  expanding  the  space  to  consider  all  fea¬ 
tures  on  all  vertices.  This  information  was  reduced  to  the  standard  deviation  of  the 


above  features,  as  it  showed  the  most  variability  across  trials.  Figure  3  depicts  differ¬ 
ences  in  graph  features  as  a  function  of  frequency  band  summarized  across  correct 
and  incorrect  trials. 


Average  Shortest  Path  Length  Degrees  Centrality 


Fig.  3.  Differences  in  graph  features  as  a  function  of  frequency  band 


4  Machine  Learning  Approach 

4.1  Dimensionality  Reduction  via  Principal  Component  Analysis. 

The  power  and  coherence  structure  feature  sets  are  high  dimensional,  with  64  power 
and  coherence  structure  features  in  each  frequency  band.  The  network  features  pre¬ 
sent  complimentary  information  with  some  overlap  across  density  thresholds.  In  order 
to  obtain  lower  dimensional  and  less  redundant  features  that  can  be  easily  utilized  to 
detect  digit  and  sentence  recall  failure,  we  use  a  common  principal  component  analy¬ 
sis  (PCA)  dimensionality  reduction  procedure  for  each  feature  set.  To  avoid  overfit- 
ting,  we  apply  an  identical  procedure  for  each  feature  set  by  selecting  the  minimum 
number  of  principal  component  features  needed  to  explain  90%  of  the  total  variance 
in  the  feature  set. 

4.2  Detecting  Digit  and  Sentence  Recall  Failure. 

For  each  subject,  detection  of  digit  recall  failure  is  done  using  statistical  models  ob¬ 
tained  solely  from  the  other  13  subjects.  The  statistical  models  we  use  are  multivariate 
Gaussian  distributions  for  each  feature  set.  The  Gaussian  classifier  output  for  feature 
set  in  a  given  trial  is  a  log-likelihood  ratio  of  recall  failure  (Class  1)  and  recall  success 
(Class  2).  The  results  are  summarized  using  the  area  under  the  receiver  operating 
characteristic  (ROC)  curves,  or  AUC,  for  each  feature  set  (Table  1).  Fusion  across 
feature  sets  is  done  by  adding  log-likelihood  ratios.  The  fused  power  feature  results 
combines  only  the  classifier  results  from  the  Beta  and  Gamma  bands,  and  the  fused 
coherence  structure  and  graph  feature  results  combine  results  from  all  four  bands. 


5  Experimental  Results 


For  this  experiment  we  report  prediction  results  obtained  using  leave  one  subject  out 
cross  validation.  Table  1  displays  the  predictive  ability  of  each  set  of  features  on  iden¬ 
tifying  mistakes  on  the  digit  repetition  portion  of  the  experiment,  and  the  sentence 
repetition  portion  of  the  experiment. 


Freq  Digit  Failure  AUC  Sentence  Failure  AUC 

Band 


Connect. 

Structure 

Graph 

Var. 

EEG 

Power 

Connect. 

Structure 

Graph 

Var, 

EEG 

Power 

Theta 

0.62 

0.49 

0.51 

0.55 

0.49 

0.39 

Alpha 

0.57 

0.54 

0.45 

0.60 

0.54 

0.40 

Beta 

0.60 

0.57 

0.60 

0.57 

0.57 

0.41 

Gamma 

0.58 

0.54 

0.59 

0.49 

0.59 

0.42 

Comb. 

0.63 

0.58 

0.61 

0.57 

0.59 

0.47 

Table  1.  Number  of  principal  components  and  area  under  ROC  curves  for  power  and  coherence 
structure  features  at  four  frequency  bands. 

In  our  analysis,  the  power,  graph,  and  structure  features  show  similar  performance 
on  the  digit  recall  task,  with  coherence  structure  consistently  performing  at  or  above 
the  other  two  feature  sets.  However,  the  power  features  lose  predictive  ability  on  the 
the  sentence  failure  task.  Additionally,  only  a  subset  of  bands  are  informative  when 
combining  across  the  three  feature  types.  Through  combining  the  three  different 
feature  sets,  we  were  able  to  achieve  a  performance  of  0.66  on  the  digit  recall  task, 
and  0.60  on  the  sentence  recall  task.  The  ROC  curve  for  the  digit  recall  task  is  shown 
below. 


Fig.  4.  Receiver  operating  characteristic  (ROC)  curve  for  best  perfonning  system,  which  fuses 
classifier  outputs  from  power  features  in  the  Beta  and  Gamma  bands,  and  from  coherence  struc¬ 
ture  and  graph  metrics  in  all  four  bands.  AUC  =  0.66. 


6  Conclusion 


In  this  paper  we  examined  the  use  of  features  derived  from  network  connectivity 
structure  to  predict  performance  on  a  digit  and  sentence  recall  task.  Additionally,  we 
compare  our  network  features’  performance  to  detection  ability  using  simple  power 
band  features  computed  over  equivalent  frequency  bands. 
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